net::ERR_ABORTED at "https://..." from a crawling program #9363
Closed as not planned
Description
Bug description
Steps to reproduce the problem:
- enclose a 'new Promise' whose body includes a 'page.goto()' then a 'page.evaluate()' inside a map that returns into a 'Promise.all()'.
const getResult = async () => await Promise.all(websites.map(site => {
const promise = new Promise(async (resolve,reject) => {
try {
await page.goto(site, {waitUntil: "networkidle2"})
const result = await page.evaluate(() => {
...
return result
})
resolve(result)
} catch (err) {
reject(err)
}
})
return promise
})
try {
console.log(await getResult());
} catch (err) {
console.log("OKOKOK", err.message);
}
// msg.text() is from a console 'on' event
Puppeteer version
^19.3.0
Node.js version
14.17.6
npm version
yarn 1.22.19
What operating system are you seeing the problem on?
macOS
Configuration file
no config file, simple node server with js scriptsRelevant log output
msg.text() Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'interest-cohort'.
msg.text() jQuery.Deferred exception: $(...).tooltip is not a function TypeError: $(...).tooltip is not a function
at HTMLDocument.<anonymous> (https://www.{site}/sites/default/files/js/js_gQZflpXLjI8oiEOVh9W7qG81ILOXaEg7B8xYt5OFDqQ.js:8924:25)
at e (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30038)
at t (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30340)
msg.text() hello ok
OKOKOK net::ERR_ABORTED at https://{site}.com
msg.text() Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'interest-cohort'.
msg.text() fancyBox already initialized
msg.text() jQuery.Deferred exception: $(...).tooltip is not a function TypeError: $(...).tooltip is not a function
at HTMLDocument.<anonymous> (https://www.{site}.com/sites/default/files/js/js_5aKlLkXJ30Pe47l8D-WslgYctBeM3RPXOkR-UpKXcJ8.js:2845:25)
at e (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30038)
at t (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30340)
✨ Done in 7.00s.
Activity
Redskinsjo commentedon Dec 6, 2022
seems to me that it is similar to #8246 with a newest puppeteer version
OrKoN commentedon Dec 6, 2022
@Redskinsjo could you please provide a minimal example that we can run? (https://stackoverflow.com/help/minimal-reproducible-example)
Redskinsjo commentedon Dec 6, 2022
sure. stackblitz and codesandbox have issues to connect to chromium, even when I install chromium locally.
anyway, please find the reproduction here :)
OrKoN commentedon Dec 6, 2022
@Redskinsjo so you are sending concurrent navigation requests to the same page and the first navigation gets aborted by the second one. Why do you think it's a bug in Puppeteer?
OrKoN commentedon Dec 6, 2022
You'd probably want to rewrite your code code as follows:
Redskinsjo commentedon Dec 6, 2022
i had to change the URL. but I am basically crawling the website and requests are sent to different URLs. Can't Puppeteer handle multiple page.goto(url) at the same time ?
OrKoN commentedon Dec 6, 2022
you should create multiple pages in that case (via
browser.newPage).